A Novel Method for Measuring the Performance of Software Project Managers

This paper is focused on providing a novel method for measuring the performance of software project managers. It clarifies the fundamental concepts of software project management, knowledge areas, life cycle phases of software project, and performance metrics. It presents some examples of processes and common performance metrics related to knowledge areas of software project management. The researcher extracts an enhanced list of performance metrics using a questionnaire that is passed to 60 experts and specialists in the field of software projects. Their responses are collected and filtered for reaching to effective performance metrics and the importance degree of each one. The researcher adapts Goal Question Metrics method to include an additional step that dedicated to calculate a performance indicator for each knowledge area of software project management. Finally, the new method has been applied on 3 real software projects to measure the performance of their managers. Measuring the performance of software project managers can be helpful in controlling and improving the performance. Keywords—software project manager; performance; measurement; metrics; indicators; Goal Question Metrics method; schedule management INTRODUCTION AND PROBLEM DEFINITION I. The knowledge and skills required for effective project managers in an Information Technology (IT) environment fall into four domains [1]: project management competencies, industry and/or business competencies, IT management practices, and general management competencies. In this paper, the researcher focuses on the project management competencies which include the IT practices. The researcher doesn’t focus on the industry and/or business competencies because they differ from one project to another. Also, the researcher doesn’t focus on the general management competencies because they are basic knowledge and skills required for any project manager. Each project management competency and its IT ramifications will be explained as a specific knowledge area of Software Project Management (SPM). Software development is a mentally complicated process. Therefore, SPM is the art and science of planning and leading software projects [2]. SPM is the on-going activities for planning, organizing, directing, and controlling progress to develop an acceptable system, i.e. conform to the quality standards within the allocated time and budget [3]. Software Project (SP) manager is a person who undertakes the responsibility of executing the SP. Software project manager is thoroughly aware of all the phases of software development process that the software would go through. A project manager closely monitors the development process, prepares and executes various plans, arranges necessary and adequate resources, maintains communication among all team members in order to address issues of cost, budget, resources, time, quality, and customer satisfaction [4]. The mismanaged projects may lead to: unfulfilled or unidentified requirements, uncontrolled change of the project scope, uncontrolled change of technology, uncontrolled risk of the project, uncontrolled subcontracting and integration, cost overruns, and/or late delivery [5]. However the number of successful SPs is few compared to the total number of software projects [6]. Figure 1 illustrates a part of a research performed by Standish Group that includes the percentage of successful, failed, and challenged software projects from year 2004 to year 2012 resulted from CHAOS manifesto [7]. Fig. 1. Project resolution results from CHAOS research [7] This research finds that around 60% of software projects were challenged or failed through the years 2004 to 2012. Because of the high percentage of failure, some research efforts are initiated to improve the performance of SP managers. This paper contributes in these efforts by providing a quantitative method for measuring the performance of SP managers at any point of time in the project life or at the end of a specific phase. The proposed method depends on an enhanced list of performance metrics and simple mathematical and statistical techniques. In addition, the proposed method depends on an adapted version of GQM method that can provide an organized method of thinking in this field. GQMI delivers an enhanced list of metrics and indicators that can be used for evaluating the performance of SP manager in order to improve and develop it. Performance indicators give a significant image about the pitfalls in the performance and then assist the management to direct the SP managers to improve their performance. The rest of the paper is organized as follows: Section II provides a background overview about the main concepts related to the research topic. Section III presents some (IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 7, No. 12, 2016 192 | P a g e www.ijacsa.thesai.org significant related work focusing on measuring the performance of SP managers. Section IV presents how to reach to the enhanced list of performance metrics and GQMI. Section V presents how to apply GQMI on real projects. Section VI concludes the paper with final remarks. The last section includes the ideas that are expected to be focused on the future. BACKGROUND OVERVIEW II. This section consists of three parts. The first part presents the main concepts related to SPM. The second part provides an overview of performance metrics of SPM. The final part gives an explanation of GQM. A. SPM Areas and Project Phases SPM is the process of planning, organizing, staffing, monitoring, controlling and leading a software project. SPM activities can be organized in nine knowledge areas [1]: integration management, scope management, schedule/time management, costs management, quality management, human resources management, communications management, risk management, and procurement management. A new knowledge area was added in PMBOK 5th edition [8, 9], which is ―stakeholder management‖.  Project integration management includes the processes required to ensure that the various elements of the project are properly coordinated [1]. It involves bringing people and things together to perform effectively. It includes the integration of functionality, data, and/or interfaces.  Project scope management involves activities to define and control what is included in the project and what is out of its scope.  Project schedule/time management is the administration and control of the finite resource of time to prevent or correct any slippages [8].  Project cost management is the planning and control required to ensure that a project is completed within the approved budget.  Project quality management involves those activities that ensure the project delivers the systems that satisfy the project objectives. The project manager must ensure that the quality activities are implemented and applied throughout the project life cycle.  Project human resources management involves those processes required to make the most effective use of the people involved in a project [1]. The project manager must identify and implement strategies to re-skill the existing IT workforce and acquire external expertise through vendors and consultants when needed [10]. Also, the project manager should establish procedures for involving the system users [11].  Project communications management involves the timely and appropriate generation, collection, dissemination, storage, and ultimate disposition of project information. Effective communication is one of the critical success factors for IS projects [12].  Project risk management includes the processes concerned with identifying, analyzing and responding to the project risks, maximizing the results of positive events and minimizing the consequences of adverse events [1].  Project procurement management includes managing subcontractors because if one of the subcontractors late, this may lead to project slippage. So, the project manager must make everything is clear to subcontractors [11]. Also, the project manager must know the legal and financial issues of subcontracting.  Project stakeholder management includes the processes required to identify the people, groups, or organizations that could impact or be impacted by the project, to analyze stakeholder expectations and their impact on the project, and to develop appropriate management strategies for effectively engaging stakeholders in project decisions and execution [8]. Each knowledge area includes a set of processes related to a specific field in SPM practices. Table I illustrates the processes required to achieve project schedule/time management, project cost management, project scope management, and project stakeholder management. TABLE I. EXAMPLE OF THE PROCESSES REQUIRED FOR ACHIEVING SPM KNOWLEDGE AREAS [8] Knowledge Area Processes Project Schedule/Time Management 1. Plan Schedule Management 2. Define Activities 3. Sequence Activities 4. Estimate Activity Resources 5. Estimate Activity Durations 6. Develop Schedule 7. Control Schedule Project Cost Management 1. Plan Cost Management 2. Estimate Costs 3. Determine Budget 4. Control Costs Project Scope Management 1. Plan Scope Management 2. Collect Requirements 3. Define Scope 4. Create WBS 5. Validate Scope 6. Control Scope Project Stakeholder Management 1. Identify Stakeholders 2. Plan Stakeholder Management 3. Management Stakeholder Engagement 4. Control Stakeholder Engagement For achieving the purpose of this paper, the researcher will focus on schedule/time management as an example of knowledge areas to explain the novel measuring method. According to PMBOK 5th edition [8] and [9, 13], the inputs, possible tools and techniques, and outputs for each process required for project schedule/time management is illustrated in table II. (IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 7, No. 12, 2016 193 | P a g e www.ijacsa.thesai.org TABLE II. THE INPUTS, TOOLS AND TECHNIQUES, AND OUTPUTS FOR PROJECT SCHEDULE/TIME MANAGEMENT [8, 9, 13] Process Inputs Possible tools and techniques Outputs Plan Schedule Management Project management plan Project charter Enterprise Environmental Factors (EEF) Organizational Process Assets (OPA) Expert judgment Meetings Analytical techniques Schedule management plan Define Activities Schedule Management Plan Scope baseline EEF OPA Decomposition Rolling wave planning Expert judgment Activity List Activity Attributes Milestone list Sequence Activities Schedule management plan Activity List Activity Attributes Milestone list Project scope management EEF OPA Precedence Diagraming Method (PDM) Dependencies Leads and lags Project Schedule Network Diagrams Project document updates Estimate Activity Resources Schedule management plan Activity list Activity attributes Resource calendar Risk register EEF OPA Expert judgment Alternative analysis Published estimating data Bottom-up estimating PM software Activity resource requirements Resource breakdown structure (RBS) Project document updates Estimate Activity Durations Schedule management plan Activity list Activity attributes Activity resource requirements Resource calendars Project scope statement Risk register RBS EEF OPA Expert judgment Analogous estimating Parametric estimating Three-point estimating Group decision making Reserve analysis Activity duration estimates Project documents updates Develop Schedule Schedule management plan Activity list Activity attributes Project schedule network diagrams Activity resource requirements Resource calendars Activity duration estimates Project scope statement Risk register Project staff assignments RBS EEF OPA Schedule network analysis Critical path method Critical chain method Resource optimization techniques Modeling techniques Leads and lags Schedule compression Scheduling tool Schedule baseline Project schedule Schedule date Project calendars PM plan updates Project documents updates Control Schedule Project management plan Project schedule Work performance data Project calendars Schedule data OPA Performance reviews PM software Resource optimization Modeling techniques Leads and lags Schedule compression Scheduling tool Work performance information Schedule forecasts Change requests PM plan updates Project documents updates OPA updates Project life cycle consists of four phases [14, 15]: project initiation, project planning, project execution, and project closure. Figure 2 illustrates the project life cycle [16]. There is a feedback between each two phases. For example, through the planning phase, the SP manager may discover that the documented project background ignores some facts in the reality. Therefore, there is a feedback between initiating the project and planning the project to redefine the project background. Each phase includes a set of processes to achieve it, but it is out of this paper scope. The performance of project managers can be effectively measured for a specific SPM knowledge area because each area includes a set of processes and skills related to a specific SPM competency. The knowledge areas are a handy way to group together theory and practical techniques [13]. It can help in discovering the weaknesses in a specific area that can be led to more attention to this area. Therefore, this is the main focus of the novel measuring method proposed in the paper. Fig. 2. Project life cycle [16] B. SPM Performace Metrics and Indicators Performance measurement is the ongoing monitoring and reporting of project accomplishments, particularly progress towards pre-established goals [17]. This process is used in project management and quality processes to determine and communicate status and accomplishments measured against specific objectives, schedules, and milestones. These measurements extend to include delivery of desired products and services to customers, whether external or internal. Performance measurement can be useful to improve future work estimates [18]. Performance measures may address: the type or level of project process conducted, the direct products and services delivered by a program, and/or the results of those products and services [17]. A metric is a quantitative measure of the degree to which a system, component, or process possesses a given attribute [19]. Metrics can be used for measuring the performance of SP manager. Performance metrics should be objective, timely, simple, accurate, useful, and cost-effective. Performance metrics can be divided into three basic categories [17]: measures of efforts, measures of accomplishments, and measures that relate efforts to accomplishments.  Measures of efforts: Efforts are the amount of resources, in terms of money, people, etc., applied to a project. Examples: The amount of money spent and the number of person-hours burned on a project.  Measures of accomplishments: Accomplishments are milestones achieved with the resources used. Examples (IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 7, No. 12, 2016 194 | P a g e www.ijacsa.thesai.org include: number of modules coded and number of deliverables.  Measures that relate efforts to accomplishments: These measures are associated with resources or cost relative to accomplishments achieved. Examples may include: amount of money expended for the portion of project completed versus the amount of money planned to be expended for this portion of work. Mike Denley proposes a list of metrics related to project cost and schedule management [20] that shown in table III. TABLE III. METRICS OF PROJECT COST AND SCHEDULE MANAGEMENT [20] Knowledge Area Metric Project Cost Management % Deviation Planned Vs. Actual Margin % Hours billed vs. project hours completed % of actual project hours completed /estimated Project hours % unplanned hours / total hours Cost Deviation From Planned Budget Estimate to Complete (ETC) (cost) Value at Completion (VAC) Budget at Completion (BAC) Project Schedule Management % or Number of Milestones Missed Deviation From Project / Program Time Schedule Planned Vs. Actual Project End Date